Improving deep neural network with Multiple Parametric Exponential Linear Units
نویسندگان
چکیده
منابع مشابه
Improving Deep Neural Network with Multiple Parametric Exponential Linear Units
Activation function is crucial to the recent successes of neural network. In this paper, we propose a new activation function that generalizes and unifies the rectified and exponential linear units. The proposed method, named MPELU, has the advantages of PReLU and ELU. We show that by introducing learnable parameters like PReLU, MPELU provides better generalization capability than PReLU and ELU...
متن کاملImproving deep convolutional neural networks with mixed maxout units
Motivated by insights from the maxout-units-based deep Convolutional Neural Network (CNN) that "non-maximal features are unable to deliver" and "feature mapping subspace pooling is insufficient," we present a novel mixed variant of the recently introduced maxout unit called a mixout unit. Specifically, we do so by calculating the exponential probabilities of feature mappings gained by applying ...
متن کاملImproving Deep Neural Networks with Probabilistic Maxout Units
We present a probabilistic variant of the recently introduced maxout unit. The success of deep neural networks utilizing maxout can partly be attributed to favorable performance under dropout, when compared to rectified linear units. It however also depends on the fact that each maxout unit performs a pooling operation over a group of linear transformations and is thus partially invariant to ch...
متن کاملUnderstanding Deep Neural Networks with Rectified Linear Units
In this paper we investigate the family of functions representable by deep neural networks (DNN) with rectified linear units (ReLU). We give an algorithm to train a ReLU DNN with one hidden layer to global optimality with runtime polynomial in the data size albeit exponential in the input dimension. Further, we improve on the known lower bounds on size (from exponential to super exponential) fo...
متن کاملExponential Stability of Linear Systems with Multiple Time Delays
In this paper, a class of linear systems with multiple time delays is studied. The problem of exponential stability of time-delay systems has been investigated by using Lyapunov functional method. We will convert the system of multiple time delays into a single time delay system and show that if the old system is stable then the new one is so. Then we investigate the stability of converted new ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Neurocomputing
سال: 2018
ISSN: 0925-2312
DOI: 10.1016/j.neucom.2018.01.084